VARIANCE COMPONENT REGULARIZATION FOR THE INFERENCE OF NEUTRAL EVOLUTION By

We represent the Brownian model for continuous traits with phylogenetic dependence as a variance components model, allowing for the novel estimation of edge weights for every branch of a tree graph. We show that maximum likelihood estimation carries an inherent bias often shrinking internal edges to zero, which is troublesome because this estimate is intended to compare covariance structures associated with neutral drift and selection. To correct this problem, we develop a new estimator that parameterizes a path between the maximum likelihood and neutral graphs leading to more stable estimation and the retention of graphical structure. Example analyses highlight the descriptive power of these estimates. We also link insights in this evolutionary context to general concerns about predicting random effects and estimating hierarchical variance components in linear mixed models when viewed in a penalized likelihood framework.